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^ ! Abstract 

Isosinglet vector-like quarks are predicted by some unified theories of electroweak and strong 

O 

interactions. We study hadron collider signatures for the production and decay of isosinglet vector- 
like quarks with charge —1/3. Previous analyses of Run I data from the Fermilab Tevatron are 
used to set lower limits of 100-200 GeV/c 2 on the mass of such quarks, depending on assumptions 
about mixing with Standard Model quarks and the mass of the Higgs boson. For future Tevatron 



used to set lower limits of 100-200 GeV/c 2 on the mass of such quarks, depending on assumptions 

in 

2^ ■ data (E c . m . = 1.96 TeV) the corresponding mass range is (100-270, 100-320) GeV/c 2 for (1, 10) 



O 

CO 

o 

On: 



fb -1 . At the CERN Large Hadron Collider (LHC) (E c . m . = 14 TeV, 100 ftT 1 ), an analysis of 
flavor-changing neutral-current decay modes should probe an h quark mass range of 100-1100 
GeV/c 2 . 
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I. INTRODUCTION 



The currently known fermions consist of quarks (u, c, t) with charge 2/3, quarks (d,s,b) 
with charge —1/3, leptons (e,/i,r) with charge —1, and neutrinos (z/ e , z/^, v T ) with charge 
0. In the Standard Model (SM), these fermions are arranged into structures that transform 
under the gauge group SU(3) c xSU(2)lxU(1)y- However, a deeper understanding of the 
particle spectrum and its pattern of charge- changing weak transitions is still unknown. 

One may try to understand the fermion spectrum and couplings by embedding the Stan- 
dard Model in a larger gauge group. For example, unified theories of the electroweak and 
strong interactions based on the group SO (10) l| can accommodate precisely this set of 
fermions within three 16- dimensional spinor representations. Larger unified groups, like 
E6 |ll S [1 , not only contain the Standard Model fermions but also predict the existence of 
new particles. The discovery of new particles predicted by higher dimensional gauge theories 
would provide insight on the organization of matter into a fundamental theory. 

A unified theory based on the gauge group E$ is phenomenologically interesting because it 
includes an enlarged lepton sector containing both massive and sterile neutrinos, an enlarged 
quark sector containing charge —1/3 isosinglet vector-like quarks (ISVLQ), and additional 



gauge bosons (e.g. Z') |3|,|^. For additional information on building low-energy models from 
the gauge group E 6 , consult Refs. □□□□□□ 

In this paper, we study weak isosinglet vector-like quarks with charge —1/3. Pair pro- 
duction of ISVLQs at high-energy hadron colliders is expected to be dominated by quantum 
chromodynamics (QCD) and thus to be precisely calculable. The deca ys o f these particles 



12 



13. 



14. 



HQ. 



We analyze 



depend on their mixing with SM down-type quarks . 11. 
the prospects for producing and detecting these exotic quarks at the Fermilab Tevatron and 
the CERN Large Hadron Collider (LHC). In particular, we consider both charged-current 
(mediated by W 1 * 1 bosons) and flavor-changing neutral-current (mediated by Z° and Higgs 
bosons) decays of the isosinglet vector-like quark. The sensitivity of these estimates to as- 
sumptions about mixing between exotic quarks and those of the Standard Model is explored. 
Related earlier studies have appeared in Refs. P, 0, [3| . 

The paper is organized as follows. In Sectioning we review relevant properties of the exotic 
quarks, and we introduce a model for their mixing with ordinary quarks. In Section IIII1 we 
briefly review constraints on the new mixing parameters. Based on these constraints, we 
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TABLE I: Exotic fermions in a 27-plet of Eg 
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propose a phenomenological parametrization of the Cabibbo-Kobayashi-Maskawa (CKM) 
matrix. Signals at the Fermilab Tevatron and the CERN LHC are treated in Sections IIVI 
and El respectively, while Section IVT1 concludes. 



II. THE MODEL 



A. Matter States Expected in Eg 

In a unified electroweak theory based on the E$ gauge group, the fundamental (27- 
dimensional) representation contains additional quarks with charge —1/3 and additional 
charged and neutral leptons. The exotic matter content of a single E 6 family is summarized 
in Table |U 0| . We assume that there are three 27-plets, corresponding to the three lepton- 
quark families. 

We adopt a "bottom-up" approach to a three-generation E 6 gauge field theory. Let 
Mi, Mi, M3 denote the masses of the three exotic charge —1/3 quarks. For simplicity we 
assume that there is a mass hierarchy between the exotic quarks such that M\ <C M2, M3. 
Hence, one of the exotic quarks will lie closer in the mass spectrum to the SM quark masses. 
In accordance with the literature 3, we denote this exotic quark as h; one should 

not confuse the exotic h quark with the SM Higgs boson, denoted H°. We assume h is the 
dominant exotic quark (relative to the two other exotic quarks) which mixes with the down- 
type SM quarks. Moreover, we assume that the exotic leptons (charged and neutral) do 
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not significantly influence SM interactions or exotic-quark signatures at the center of mass 
(CM) energy of the Tevatron or the LHC. Production of exotic leptons, if kinematically 
allowed, should only proceed via the electroweak sector of the theory, and thus should be 
suppressed with respect to exotic quark production. Under these assumptions, we have 
effectively reduced the E 6 model at the CM energies of the Tevatron and the LHC to a 
model which contains the SM along with a single down-type exotic quark. 

The exotic quark, h, is a down-type quark, but unlike the SM quarks, the ISVLQ is a 
singlet under the SU(2) L factor of the SM gauge group. The singlet nature of the down-type 
ISVLQ introduces new mixings and interactions between the quarks. In the remainder of 
this section, we construct these interactions and explore their influence on the CKM matrix. 

In this paper, the SM fermions are labeled by a generation index % {% — 1, 2, 3) and we 
label the h quark by the index value 4. The indices (i, j, k) run from one to three and the 
indices (/,m, n) run from one to four. The "L" and "R" subscripts are employed to denote 
the left- and right-handed components of fermion fields (i.e., -ul,r = Pl,r« = [|(1 T 7 5 )]« 
in our notation). To facilitate our discussion it is useful to define the left-handed quark 
doublet, and the left- and right- handed quark field vectors: 

Qi = (u Li , d Li ) T , U A = (u A , c A , t A ) T , D A = (d A , s A , b A , h A ) T , (1) 

where A = L,R. We denote the flavor eigenstates via the "0" superscript. 

In the flavor basis, the kinetic piece of the quark Lagrangian is obtained via minimal 
coupling 

+gw;j^ + + g w; + g zir z + eA,r EM) (2) 



where the Jem-, J\v±, an d Jz are the electromagnetic, charged-weak, and neutral- weak 
current operators. In the weak eigenbasis these currents take the form 




where sw and cw are the sine and cosine of the weak mixing angle, C£ = — QaSyy and 
Cr = -Q a s 2 w (a = u,d). 

In the Standard Model, quark masses are generated when the Higgs doublet acquires a 
non-zero vacuum expectation value (VEV); the resulting mass terms in the Lagrangian have 
AJl = 1/2. Unlike the SM, theories with exotic E$ quarks have AJl = mass terms. In 
the ISVLQ model, these Dirac mass terms result from the left-handed h field (recall h is an 
SU(2) L singlet) pairing with right-handed quark fields. We write the quark mass terms of 
the Lagrangian in the compact form 

C mass = -UlM u Ul - D° L M d D° R + h.c, (4) 

where A4 U and M. d are the up-type quark and down-type quark mass matrices, respectively. 
The up-type quark mass matrices mimic those of the SM; however, the down-type quark 
mass matrix has been enlarged. The down- type quark mass matrix contains both A/l = 1/2 
and A/ L = entries. 

It is important to distinguish the A/l = 1/2 and A/l = mass terms since they may 
arise from fundamentally different scales. The AJl = 1/2 mass terms are derived from 
spontaneous symmetry breaking (SSB) of the SM Higgs at the electroweak scale; hence, 
these mass terms should be on the order of the electroweak scale or smaller. On the other 
hand, the A/ L = mass terms do not result from electroweak symmetry breaking (EWSB); 
therefore, they may be of any scale, possibly the unification scale. We distinguish A/ L = 1/2 
and AI L = mass terms in the down-type quark matrix by lower- and upper-case letters, 
respectively, 

' m dd m ds m db m dh ^ 
, id m sd m ss m sb m sh 

M = . (5) 

m bd m bs m bb m bh 

K M hd M hs M hb M hhj 

We investigate the production and decay of an exotic E 6 quark at the Fermilab Teva- 
tron and the CERN Large Hadron Collider. To understand exotic quark decay we must 
determine the branching ratios of these exotic fermions to SM particles. Hence we must de- 
termine the analogue of the CKM matrix and the no-longer trivial flavor- changing neutral 
current (FCNC) matrix for this theory, and we must constrain elements of these matrices by 
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experimental data. To determine the CKM and FCNC matrices we must recast our theory 
in the mass eigenbasis. 

Each quark mass matrix in this theory may be diagonalized by two unitary transforma- 
tions, denoted L and R, 



diag 



L d ) ] M d R d , (-M") diag = {L u ) j M u R u 



(6) 



where the u and d unitary matrices are 3x3 and 4x4, respectively. The fields in the mass 
basis are related to the fields in the flavor basis by the L and R transformations: 



D L = (L d yD° L , D R = (R d yD° R 

u L = (L u yui , u R = (R u yui 



(7) 



Using the relations between the mass and flavor eigenstates, we recast the current operators 
of Eq. (JHJ) in the mass eigenbasis: 



7" M — —V 
~ y/2 



= v* 

' a/2 



D n -fP L Ui 



(8) 
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where 



3 t d d 1 d* d 

Vin = ^2(L u ) ik L kn > X nm = C L 5 nm + -L 4n £ 4m , 

k=l z 



(9) 



denote the CKM and FCNC matrices (i = 1,2,3, and m,n = 1,2,3,4), respectively. One 
obtains the expression for the FCNC matrix by transforming the coupling of the left-handed 
down-type quarks to the Z° from the weak basis to the mass basis, 



c w * 

cw 
c w 



D m ' [ C d 5 m i n > + -L d n,L d m , ) 7 M P L -D n ' 



(10) 



Note: If the ISVLQ were an up-type quark, then the Z° boson would have a corresponding 



FCNC coupling to the up-type quarks of the form, C^S mn — \V£ n L\ m . 
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This theory differs from the SM in the structures of its CKM and FCNC matrices. In 
the SM, the CKM matrix is a 3 x 3 unitary matrix, while in the ISVLQ model it is a 
3x4 matrix (non-unitary). Hence the "unitarity triangle" approach to determining CKM 



parameters must be abandoned in favor of a "unitarity quadrangle" [17(. In the SM, the 
FCNC matrix is diagonal, hence, there are no tree-level flavor-changing neutral couplings. 
In the ISVLQ model this is no longer true; in essence, one abandons the Glashow-Iliopoulos- 
Maiani (GIM) mechanism [18] which suppresses flavor- changing neutral currents. In Section 
3, we investigate the structure of the CKM and FCNC matrices [both matrices are related 
to the L d matrix]. 



B. Feynman Rules 

As explained in Section III A| we consider a simple Eg inspired extension of the SM, in 
which one down-type ISVLQ interacts with the SM particles. In Fig. ^ we summarize the 
effects of this ISVLQ on the quark interactions with SM gauge and scalar fields. In this 
paper, all calculations are performed in the Feynman-'t Hooft gauge. 

The addition of an ISVLQ not only induces Z°-mediated tree-level FCNC interactions, 
but also induces tree- level FCNCs mediated by the Higgs, H°, and Goldstone, <fi z , bosons. 
Therefore, in certain instances, the branching ratio of the h quark to a Higgs boson is large. 



III. CONSTRAINTS AND THE PARAMETRIZATION OF THE CKM MATRIX 
A. Physical Parameters 

It is useful to parametrize the complex non-unitary 3x4 CKM matrix in terms of physical 
parameters. The number of physical parameters of a theory may be determined by analyzing 
its symmetries. Consider the up- and down-type quark mass matrices in Eq. each mass 
matrix is complex, hence there are (9 + 16) x 2 = 50 real parameters. To determine the 
number of physical parameters let us turn off the mass matrices. Setting the quark mass 
matrices to zero, the theory admits a global quark symmetry, 

GgioUM u = M d = 0) = U(3) Q x U(4) d - x U(3) a x U(l) fc , (11) 
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FIG. 1: Feynman rules for a model with one ISVLQ down quark. The indices i, j = 1, 2, 3 and the 
indices n, m = 1, 2, 3, 4. 

where the subscript Q, d, u, and h correspond to the left-handed SM quarks (transforming 
as 2 under SU(2) L ), right-handed down-type quarks, right-handed up-type quarks, and left- 
handed ISVLQ, respectively. This global symmetry contains 9 + 16 + 9 + 1 = 35 real 
parameters. If we turn the quark mass matrices back on, then the global symmetry is spoiled 
and we are left with a remnant global U(l) quark symmetry. Consequently, there are 50—35+ 
1 = 16 physically significant real parameters: 7 quark masses, and 9 mixing parameters (6 
angles and 3 phases). One uses the 9 physical mixing parameters to parametrize the unitary 
matrices of the theory (L d , R d , L u , and R u ) and hence the CKM and FCNC matrices. 
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Assuming that we have solved for the up-type quark mass eigenstates, then the CKM 
matrix expression Q reduces to 

Uin = L ini (12) 

where i = 1,2,3 and n— 1,2, 3, 4. If we parametrize the 4x4 unitary L d matrix, we recover 
the CKM matrix by restricting to the upper 3x4 sub-matrix. We recover the FCNC matrix 
by selecting appropriate combinations of the Lf m entries [see Eq. (£]J)]. 



B. Constraints on h Quark Mixings 

The L d matrix may be parametrized by nine physical parameters; however, the ISVLQ 
model does not specify the magnitude and pattern of the resulting interactions. Because 
we are interested in the production and decay of h quarks, constraints on the magnitude of 
the new mixings between the h quark and the SM quarks are important. To determine the 
magnitude and pattern of these and other interactions (charged- current and flavor-changing 
neutral-current), experimental data must be examined. Using experimental constraints on 
elements of the CKM and FCNC matrices, one may infer the structure of the L d matrix 
[consult Eq. ©]. 

Data from precision electroweak experiments and low-energy flavor- changing neutral pro- 
cesses help to constrain CKM and FCNC matrix elements. We use recent constraints on 
CKM and FCNC matrix elements obtained by Refs. QQ. Fo r addition*, info— on 



constraints of the CKM and 
to studies discussed in Refs. 



CNC matrix elements in an ISVLQ model, we refer the reader 
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12 



14, 



17 



21 



22j. In Ref. [3], the following constraints on 



the magnitude of L d matrix elements were obtained: < \L d 4 \ < 0.087, < \L d 4 \ < 0.035, 
< \L d A \ < 0.041, and 0.998 < IL33I < 1. The constraint on the magnitude of L d 3 is required 
by the agreement of Rb with experiment. Constraints on the magnitude of the Lf 4 , L d 4 , and 
L d A elements are obtained from the observables |<5mB|, l^mgj, e, e'/e, the branching ratios 
for b — > se + e~, b — > K + — > -n + vi>, Kl — > and the CP asymmetry a^K s - 

Ref. also finds that, at present, there are no restrictions on h quark masses below 1 
TeV/c 2 . Using the constraints on the L d matrix in addition to constraints on the mixing 
between SM quarks, we can parametrize the L d matrix. 
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C. Wolfenstein Parametrization 



There are a number of ways to parametrize the L d matrix, and hence the CKM matrix. We 
adopt a Wolfenstein-inspired 



23| parametrization. We require the elements of the principal 
diagonal and the sub-diagonal directly above the principal diagonal to be real. In analogy 
with the phase in the SM Wolfenstein parametrization, the two new phases of the matrix 
are assigned to the Lf 4 and L d A elements. The presence of additional phases in the ISVLQ 



model may lead to additional sources of CP violation; for studies of CP vio 



25 



ation arising 



26 



27]. 



from the new phases of the L d matrix the reader should consult Refs. 

We write Lf 4 = Aue iuJ2 X 2+ni4 , and L d 2A = Ane iuJ2 A 2+ ™ 24 , where u 2 and u 3 are the two 
new phases, v and k (z/, k ~ 0(1)) are two of the three new angles, the Wolfenstein Ll A" 
parameter is inserted for convenience, and and ri24 are integers greater than or equal 
to 0. If the h quark were to mix strongly with either the u or c quarks [i.e., if L d A or 
L d A were of O(X)], then current SM mixing contained in the first two rows of the L d and 
CKM matrix would be affected 28[. The third new angle, £, parametrizes the mixing of the 
ISVLQ with the third generation of quarks. We parametrize the L d A matrix elements as £A 2 
where £ ~ 0(1). The magnitude of the Lf A , L d A , and L d 4 elements are consistent with the 
constraints on the L d matrix outlined in Section IIII Bl (£ < 1 from bound on L\ 

Using this framework, we create a preliminary sketch of the L d matrix, 



4a) 
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td 
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Td 
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Td 

L 42 



Td 

-^43 



Aue iuJ2 A 2+ni4 ^ 
Ane iuJ3 X 2+n24 

£A 2 
1 - c 4fe A 4 + I. 



lib 



(13) 



where the c nm are multiplicative factors of 0(1), and the l nm are higher-order contributions 
in A to the L d m elements (e.g. in the L d 2 element, l ts ~ C(A 3 ) since c is A 2 is of second order 
in A). Using the unitarity of the L d matrix and additional assumptions regarding the size 
of particular elements, one may obtain a fully parametrized expression for the L d matrix. 

As previously mentioned, we expect the h quark to mix predominately with the third 
generation. Therefore, to simplify the parameter space we assume that the mixing of the 
h quark with the first two generations is small. Namely, we assume that nu and ri2i are 
greater than the order of the L d matrix parametrization. Parametrizing L d to 0(X 5 ), one 
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obtains the following matrix, 



1 - \X 2 - |A 4 A 

-A 1 - ±A 2 - (| + \A 2 ) A 4 

A {I- fie~ iuJl ) A 3 -AX 2 - A (-\ + /le^) A 4 1 
A£A 4 



A/ze^A 3 
AX 2 



(±£ 2 + \A 2 ) A 4 £A 2 

-^a 2 i - \e\ 4 j 



(.14) 



V 



In this parametrization of the L d matrix, the phase u\ = —7 and the angle fi = a/ P 2 + V 2 - 

Using this form of the L d matrix and Eq. (JHJ), the dominant new charged- current coupling 
is between the top and h quarks. The strength of the t-h charged- current interaction is 
g(2 -1 / 2 )£A 2 . In the ISVLQ model, tree-level FCNCs are expected between all four down-type 
quarks; however, we find that tree-level flavor- changing neutral currents between SM quarks 
are disfavored (0(X 6 )). Interactions between the h quark and SM down-type quarks may be 
sizable; in fact, interactions between the h quark and b and s quarks are of 0(X 2 ) and 0(X 4 ), 
respectively. In particular, the h-b interaction is left-handed with strength g(2c^)^ 1 ^,X 2 [to 
0(X 5 ) in the parametrization]. 

The form of the L d matrix in Eq. (|T4*|) is just one of many possibilities arising from the 
constraints in Section IIII Bl For example, the strength of the charged-current interaction 
between the h quark and the u or c quarks may not be negligible {e.g. n u or n 2 4 may equal 
zero). If these charged-current interactions are relevant, then they will contribute to the 
"td" and u ts" elements of the CKM matrix. These additional charged-current interactions 
lead to corrections to the ltd and lt s terms of the L d ± and L d 2 elements. In the remainder of 
this paper, we use the L d matrix in Eq. (fT4*j) . 

IV. hh PRODUCTION AND DECAY SIGNATURES AT HADRON COLLIDERS 

In this section we investigate the prospects for h quark observation at the Fermilab Teva- 
tron and the CERN Large Hadron Collider. In pp and pp collisions, hh production proceeds 
predominantly through QCD interactions; therefore, in the remainder of our analysis, we 
suppress contributions to hh production from electroweak processes. 
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A. hh Production 

To obtain a basic understanding of the hh production rate and the h quark mass reach 
at hadron colliders, we plot [see Fig. |2| the tree-level hh pair production cross section at the 
Fermilab Tevatron (y/s = 1.96 TeV) and CERN LHC (\/s = 14 TeV) as a function of the 
h quark mass, calculated using the CTEQ5L structure functions j^]. The curves in each 
of these plots correspond to different choices of the QCD Q scales (Q 2 = M§, m|, (2m/ l ) 2 , 
and s). For small h quark masses the cross sections for each of the Q scales are comparable; 
however, at h quark masses above ~ 200 GeV/c 2 there is a large discrepancy between the 
cross section with the fixed Q scale (Q 2 = M§) and the dynamic Q scale (Q 2 = m|, (2m h ) 2 , 
and s) . For our analysis at the Tevatron we use a Q scale which is set by the CM energy of 
the subprocess, and at the LHC we use a Q scale equal to twice the h quark mass. 

Using Fig. El and projected integrated luminosities, a 'back of the envelope' upper limit 
for the h quark mass reach is obtained. For an integrated luminosity of (1, 10) fb _1 at 
the Tevatron, the largest h quark mass reachable (producing at least one hh event) is ~ 
(490, 550) GeV/c 2 . At a luminosity of 100 fb _1 at the LHC, the largest accessible h quark 
mass reachable is ~ 2500 GeV/c 2 . These estimates assume perfect event detection and they 
assume there are no complications arising from the decay of the h quarks. 

Though Fig. |21 is instructive, one must take into consideration final-state signatures and 
backgrounds to determine a realistic h quark mass reach. The h quark is expected to be 
an unstable particle and hence it will decay to SM particles. The three dominant h quark 
decay processes are h — > tW~, h — > bZ°, and h — > bH°. Since each decay mode of the h 
quark depends on the new mixing parameter £, the cross section for h quark production and 
decay must also depend on the the mixing parameter. 

We calculate the cross sections for hh pair production in conjunction with decay of 
the h quark into a SM quarks and gauge bosons. To facilitate this analysis we use the 
CompHEP [30] software package to calculate the tree-level cross sections for hh production 
and decay at the two collider facilities. For the remainder of this section we shall restrict 
our analysis to the Fermilab Tevatron, reserving a brief discussion of the LHC for Section IVl 
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FIG. 2: Plot of the /i/i production cross section as a function of the h quark mass at (a) the Fermilab 
Tevatron, ^/s = 1.96 TeV, and at (b) the CERN LHC, = 14 TeV. The curves in each of these 
plots correspond to different choices of the QCD scale (Q 2 = M§, m|, (2m/,) 2 , and s), where s is 
the square of the subprocess CM energy. In the Tevatron plot, the curves for Q 2 = (2m/,) 2 and 
Q 2 = s overlap, while in the LHC plot we omit the Q 2 = s curve since the CTEQ5L structure 
functions |29j are not defined beyond 10 TeV. 



B. h Quark Width 

To calculate the cross sections for hh production and subsequent decay, we must first 
calculate the width of the h quark. The total width of the h quark is the sum of the h quark 
partial widths; the decay processes contributing to the partial widths may be separated into 
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three (decay) channels: h — > u\ W , h — > di Z°, and h — *> di H°. To leading order the partial 
widths for the three decay channels are given by the expressions, 



G F \ ( c 2 w M 2 z , > 

m h J 



r(/i^^H°) = r^i^iLlHLliVK^^M^GK^^Ml), (15) 

where p* is the center of mass momentum, and 

F(m 2 , m 2 , M 2 ) = ml + ml - M 2 + — (m{ - m 2 . - M 2 ) (m 2 - m\ + M 2 ) 
G(m 2 , m 2 , M 2 ) = 2 1 Um 2 m 2 + (m 2 + mjj) (m 2 + m\ - M 2 ) 1 . (16) 

The center of mass momentum may be expressed as p*(mi, m-i, M) = 2^"(-^( m i? m 2' ^ 2 )) 5 ) 
where A(a, b, c) = a 2 + b 2 + c 2 — 2a6 — 2ac — 26c. The partial widths for h quark decays (via 
the W~, Z° and H° channels) are shown in Fig. Efa). 

In Fig. 02b), we plot the branching ratio for the h quark decaying via Z°, W~, and H° 
channels. In this plot the mixing parameter, £, is 1, the mass of the Higgs Boson is 150 
GeV/c 2 , and the mass of the h quark is varied from 100 to 1000 GeV/c 2 . The Z° decay mode 
is the dominant decay channel for h quark masses below 200 GeV/c 2 ; however, the branching 
ratio to this mode quickly diminishes as the H° and W ± decay modes become kinematically 
accessible. Since the charged- current u and c couplings to the h quark are small [see Eq. (|14))]. 
the branching ratio of the h quark to W~ is suppressed until m^ > Mw + m t . In the large h 
quark mass limit, the ratios of the Z°, H°, and W~ partial widths are |L4 4 | 2 :|L4 4 | 2 :2. Using 
the L d matrix paramterization in Eq. (|14p. these ratios are (1 — £ 2 A 4 ) : (1 — £ 2 A 4 ) : 2. 



C. Constraints on ISVLQ model from b' searches 

Before we discuss specific signatures of hh pair production and their corresponding cross 
sections, we investigate limits on the h quark mass from previous experiments. In particular, 
we determine implied limits from the first run of the Fermilab Tevatron. 

In Run IB of the Fermilab Tevatron {^fs = 1.8 TeV), the CDF and D0 collaborations 
acquired 86.47 pb" 1 and 84.5 pb _1 of data, respectively 31]. Though a specific analysis of 
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FIG. 3: (a) Plot of the h quark partial widths as a function of the mixing parameter £ (£ = 
0.4,1.0,1.6) and h quark mass, m^. (b) Plot of the h quark branching ratio for a fixed mixing 
parameter value £ = 1 and a variable h quark mass, m^. In each of these plots, the Higgs boson 
mass is taken to be 150 GeV/c 2 . 

the ISVLQ model using the Run IB data was not considered, we can infer limits on the h 



quark mass from b' searches [3J 
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Q. Unlike the ISVLQ h quark, the b' is the charge 
— 1/3 member of a fourth quark generation, (t'b') T . Since the b' is a member of a doublet, 
the GIM mechanism is preserved and, therefore, flavor-changing neutral decays of the b' 
are forbidden at tree-level. Consequently, flavor-changing b' decays occur via higher-order 
interactions; they are expected to be suppressed relative to tree-level interactions. 

In these b' studies, the CDF and D0 collaborations searched for a fourth-generation b' 
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quark by looking for events where the b' decays via a flavor- changing neutral interaction. In 
particular, both the CDF and the D0 collaborations searched for a V quark in the following 
mass regions: (1) my < M z [see Ref. 2^1 ] and (2) my > M z + m b but my < m t and 
my < m t ' [see Ref. j33j]]. In the former mass region, the b' decays to a photon and a SM 
down-type quark, while in the latter mass region the b' decays predominately to a Z° boson 
and a SM down-type quark. Note that in the second mass region, tree-level charged-current 
decays to light, up- type quarks are present; however, they are doubly Cabibbo-suppressed 
by the small coupling between the b' quark and the light up-type quarks. 

In this paper, we are interested in an ISVLQ h quark with a mass greater than that of 
the Z° boson. In the top panel of Fig. we reproduce the 95% confidence limit (solid line) 
on the cross section for a b' quark of Ref. |33j]. In the bottom panel of Fig. 01 we plot the 
cross section for the hh production multiplied by the branching ratio for each h quark to 
the Z° mode. The four curves in this plot correspond to the aforementioned choices of the 
QCD Q scale. Using the \/I Q scale, one finds that for a mixing parameter of £ = 1 and 
a Higgs mass of 150 GeV/c 2 the Run I data exclude a h quark at the 95% confidence level 
in the 100 - 200 GeV/c 2 mass range. For a Higgs mass of 115 GeV/c 2 , the Run I analyses 
imply that an h quark is excluded in the mass range of 100 - 185 GeV/c 2 . In the following 
sections, we will discuss the effect of the £ mixing parameter on the h quark mass reach of 
current and future hadron colliders. 



D. hh Production and Decay at the Tevatron 

At the CM energy of the Fermilab Tevatron (y^i = 1.96 TeV), hh pair production is 
dominated by subprocesses in which a quark from a proton and an anti-quark from the 
anti-proton annihilate via the strong interaction. Contributions from subprocesses involved 
in gluon-gluon fusion are suppressed by the gluon density of the proton, and sea quark 
contributions are suppressed by the sea quark distributions of the proton. 

Once an hh pair is produced, each isosinglet quark decays to a SM quark and an associated 
gauge or scalar boson. Though subsequent decay and hadronization of these particles is 
expected, we do not explicitly generate the matrix elements for these individual processes. 
Rather, we generate the matrix elements for hh pair production and the primary decay of 
the h quarks into a heavy quark (bottom or top) and an associated gauge or scalar boson. 
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FIG. 4: (Top) Exclusion plot for the b' search from run IB of CDF 33]. (Bottom) Plot of the 
cross section for h quark pair production and subsequent decay into two b quarks and two Z° 
bosons. In this plot the mixing parameter £ = 1, the Higgs mass is 150 GeV/c 2 , and the curves 
correspond to different values of the Q 2 scale. 

We do not consider processes in which the primary decay of the h quark results in a light 
quark. Processes in which the h quark decays to a W~ and a light quark are disfavored by 
our choice of CKM matrix, [see Eq. ()14j) ]: moreover, processes in which the h quark decays 
to a Z° or a H° and a light down-type quark often lead to complicated, multi-jet event 
topologies. 

For the Tevatron, the hh production and decay schemes considered in this paper are: 
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1. pp -> hh-> ti W+W - , 

2. pp-^hh^ bb Z°Z°, 

3. pp^hh^ bb H°H°, 

4. pp^tih^ bb H°Z°, 

5. pp^hh^ tb W~Z° + a W+Z°, 

6. pp^hh^ tb W-H° + ft W _ H°. 

Promising final-state signatures and backgrounds for these processes are discussed in Sec- 
tions IIVD II and IIV D 21 respectively. For the time being, we are interested in the cross 
sections for these six hh production/decay schemes and which, if any, of these schemes 
provides the best channel (s) for an h quark search at the Tevatron. 

TABLE II: We require that the h quark decay products pass basic pseudorapidity (77), transverse 
momentum (pr), and angular separation (AR) cuts. Note that the angular separation cut is only 
applied to the bottom quarks and not to the top quark or the bosons (gauge nor scalar). 

Parameter Minimum Value Max Value 
% -3.0 3.0 

AR(6i,6 2 ) 0.4 
p b T 25.0 



To begin, we impose a loose set of cuts to ensure that the produced hh events conform to 
basic geometry and event selection requirements of the detectors. These cuts are summarized 
in Table HU We impose a loose cut on the pseudorapidity r\ of the bottom quark, 1 77^ | < 3. 
Pseudorapidity is defined as rj = — logtan(#/2), where 6 is the angle between the particle 
being considered and the undeflected beam. In addition to the pseudorapidity cut, we 
impose a jet separation cut, AR = yj (A7/) 2 + (A0) 2 , to ensure that there is adequate jet 
separation for detection. Finally, we impose a cut on the transverse momentum, px, of the 
bottom quark. Because the bottom quark is a decay product of a much heavier h quark, one 
expects the bottom quarks to be "hard" (high momentum) and to have substantial transverse 
momentum which scales with the mass of the h quark. Moreover, because previous data from 
the Fermilab Tevatron appear to exclude an h quark up to ~ 200 GeV/c 2 (when M# = 150 
GeV/c 2 ), a "hard" cut on the transverse momentum of the b quark will merely reduce the h 
quark signal in a previously excluded region (i.e. 100 - 200 GeV/c 2 ). Therefore, we impose 
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a lower bound of 25 GeV/c on the transverse momentum of each b quark. As we shall discuss 
in Section IIV D 21 the px cut and the AR cut on the b quarks help to reduce backgrounds 
for these hh pair production processes. The top quark and the gauge bosons are unstable 
particles; therefore, we do not impose any constraints on the these particles. When we 
discuss event signatures in Section IIV D 11 we will impose cuts on the decay products of 
these particles. 

In Fig. we illustrate the effects of these cuts on the pp —>■ hh —>■ bbZ°Z° signal. In 
Fig. Efa), we plot the pp — > hh — > bbZ°Z° cross section for the following cuts: "no cuts" 
(unconstrained), "base cuts" [see Table ITT] . "tighter pseudorapidity" cut (|^| < 1.5 GeV/c), 
"looser transverse momentum" cut (p? > 15 GeV/c), and "tighter pseudorapidity/looser 
transverse momentum" cut (|t7&| < 1.5 and pt > 15 GeV/c). In Fig. EJb), we present each 
of these cross sections divided by the "base cuts" cross section. Above an h quark mass of 
180 GeV/c 2 , loosening the transverse momentum cut from 25 GeV/c to 15 GeV/c on each b 
quark increases the cross section by less than 10%. Tightening the pseudorapidity cut from 
3 to 1.5 on both b quarks reduces the cross section by roughly 20%. Modified constraints on 
the b quark pseudorapidity may be used to limit the reduction in the cross section to less 
than 20%. For example, one may require at least one b quark to be "tight" (rjb < 1.5) and 
the other b quark to be "loose" (rjb < 3). 

The cross sections for hh production and (primary h quark) decay are shown in Fig. |UJ 
The new mixing parameter £ is set to 1 and the h quark mass, rrih, runs from 100 - 500 
GeV/c 2 . The cross sections for each of the primary decay modes fall rapidly as the h quark 
mass is increased. Because of the transverse momentum cut on the b quarks, primary decay 
modes containing at least one b quark exhibit exaggerated rounding of peaks in the cross 
section. Cross sections for modes in which the decay products of the h quark are more 
massive than the h quark are suppressed. In particular, the cross section for the tiW + W~ 
mode is suppressed until rrih > m t + M w . As a result of the suppression and the CM energy 
of the Tevatron, cross sections for modes in which at least one h quark decays to a top quark 
are small (less than 10 fb). 

At an integrated luminosity of (1, 10) fb -1 , the largest accessible h quark mass (at £ = 1 
and Mn — 150 GeV/c 2 ) for any of the primary h decay modes is ~ (420, 500) GeV/c 2 
(modes produce one signal event). The most promising of these primary decay modes are 
the bb Z°Z° and bb H°Z° modes. Each of these modes has a "large" cross section below an h 
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FIG. 5: Effect of cuts on the pp — > hh — > 66Z°Z° cross section where the mixing parameter £ = 1, 
and the Higgs mass M H = 150 GeV/c 2 . (a) Plot of a(pp hh -> 6/iZ°Z°) for five different choices 
of 6 quark cuts: (solid) iVo Cuts; (long dash) i?ase Cuts, see Table HH (short dash) Reduced pr, 
base cuts with p h T > 15 GeV/c; (dots) Tighter rj, base cuts with |%| < 1.5; (dash-dot) Tight -q and 
Loose pt, base cuts with the tighter rj and a looser pp. (b) Plot of the cross sections relative to 
the "Base Cuts" cross section. 

quark mass of 300 GeV/c 2 , and their respective cross sections are comparable to the other 
modes above an h quark mass of 300 GeV/c 2 . As discussed in the signatures section, these 
modes can give rise to clean distinctive signatures at the Tevatron. On the other hand, 
modes in which at least one h quark decays to the tW~ channel have relatively small cross 
sections at the Tevatron. Subsequent decay of the top quark often leads to complicated final 
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FIG. 6: Cross sections for hh production and primary decay at the Fermilab Tevatron. In this plot 
the Higgs mass is Mh = 150 GeV/c 2 , £ = 1, and Q = y/§. 



states. 

Effect of the £ Mixing Parameter on Cross Sections: 

Before we discuss signatures of hh pair production at the Tevatron, we address the depen- 
dence of the cross sections on the mixing parameter, £, and on the Higgs mass. In Fig. [7f a) 
and Fig. Efc), we plot the pp — > hh — > bbZ°Z° and pp — > hh — > 65H°Z° cross sections for 
a Higgs mass of 150 GeV/c 2 and for four choices of the £ parameter, £ = 0.02, 0.2, 1, 
2 35j]. In Fig. |7fb) and Fig. E^d), we plot the £ = 0.02, 0.2, 1, 2 cross sections divided by 
the £ = 1 cross section. In both the bb Z°Z° and the bb H°Z° modes, the cross sections are 
weakly dependent on the £ parameter. In the h quark mass range of 100 - 500 GeV/c 2 , the 
change in the £ parameter causes no more than a 10% change in the bbZ°Z° cross section 
and no more than a 25% change in the bb H°Z° cross section. Therefore, reasonable changes 
in the £ parameter should not significantly impact the mass reach capability at the Fermilab 
Tevatron. In the remainder of this paper, we restrict our analysis to £ = 1. 
Effect of Higgs Mass on Cross Sections: 
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FIG. 7: Effect of the mixing parameter ^ on the "primary decay" cross sections. The curves in 
each of these plots correspond to four different choices of the £ mixing parameter (£ = 0.02, 0.2, 
1.0, and 2.0). In each of these plots, the Higgs mass is 150 GeV/c 2 . (a) Plot of the cross section 
for pp — > hh — > bb Z°Z° as a function of h quark mass, (b) Plot of each of the bb Z°Z° cross sections 
relative to £ = 1 cross section, (c) Plot of the cross section for pp — > hh — > 66H°Z° as a function 
of h quark mass, (d) Plot of each of the 66H°Z° cross sections relative to £ = 1 cross section. 

In Fig. |Sl we plot the bb Z°Z° and bb H°Z° cross sections for three choices of the Higgs boson 
mass: Mh = 115, 150, and 175 GeV/c 2 . At tree level, the 66Z°Z° cross section depends on 
the Higgs mass through the width of the h quark. In Fig. |Sf a) a heavier Higgs boson leads 
to an enhancement in the bb Z°Z° cross section. This can be understood as a suppression of 
the H°cfj decay channel in the h quark branching ratio [see Fig. E^b)]. 

In the 66H°Z° cross section the Higgs boson is taken as an "external particle" in the 
Feynman diagrams. Therefore, the Higgs boson mass enters the cross section through the 
phase space integration and the expression for the h quark width. In Fig. IBJb) one observes 
that by reducing the Higgs boson mass to 115 GeV/c 2 , the cross section is enhanced for the 
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FIG. 8: Effect of the Higgs mass, Mh, on the primary decay cross sections. In each of these 
plots the mixing parameter, £, is fixed at 1, and the curves correspond to different Higgs masses 
(Mh = 115, 150, 175 GeV/c 2 ). (a) Cross sections for pp — > hh — > 66Z°Z° as a function of h quark 
mass, and (b) Cross section of pp — > hh — > 66H°Z° as a function of h quark mass. 

mass range of 100 - 300 GeV/c 2 . On the other hand, if we increase the mass of the Higgs 
boson to 175 GeV, the cross section is reduced in the same region. Above an h quark mass of 
300 GeV/c 2 , changes in the Higgs mass result in small changes to the 66H°Z° cross section. 
The negative correlation between the size of the 66H°Z° cross section and the mass of the 
Higgs boson below 300 GeV/c 2 follows from the suppression of phase space by the Higgs 
mass. In the remainder of this paper, we consider two Higgs boson masses, 115 GeV/c 2 and 
150 GeV/c 2 . 
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1 . Signatures 



In this section, we investigate final state signatures for hh pair production at the Tevatron. 
To facilitate the discussion, we categorize signatures based on the primary decay modes of 
the hh pair. 

Below an h quark mass of 300 GeV/c 2 , the dominant primary decay modes are the bb Z°Z° 
and bb H°Z° modes. Above an h quark mass of 300 GeV/ c 2 , each of the primary decay modes 
[see Fig. |B] are comparable in size. Therefore, a number of signatures arising from the decay 
of the 66Z°Z° and the 66H°Z° modes will be important for an hh search. Though cross 
sections from the other decay modes are comparable to the bb Z°Z° and the bbH°Z° modes 
when the h quark mass is greater than 300 GeV/c 2 , signatures arising from these modes are 
often challenging experimentally. 

We do not consider the tbZ W~ (6tZ°W+), t6H°W~ (6tH°W+), and ttW+W" modes 
because the decaying t quark produces a b quark and another W ± boson. As a result, 
signatures arising from these modes have at least two more final-state particles than the 
bb Z°Z° mode. At the Tevatron, small cross sections in conjunction with complicated event 
topologies, small branching ratios, and detector effects lead to little or no mass reach for 
these modes. 

For example, signatures arising from the tiW + W~ mode have complicated final states. 
In this mode, each t quark decays to a b quark and a boson: (5W + ) t (6W~) t W + W~. In 
our notation, the "t" subscript in "(•)*" indicates that the quantities enclosed in the paren- 
theses have an invariant mass of the top quark, m t . Each of the 4 W ± bosons decay to either 
a quark/ ant i- quark pair (hadronically) or to a charged lepton/neutrino pair (leptonically) . 
In the detector the quark/anti-quark pair hadronize into two jets (jj), and the charged 
lepton/neutrino pair manifest as a lepton 36] and missing transverse energy (/ ± E T ). Two 
possible final state signatures are: u bb(jj)w{jj)w(jj)w(.jj)w" an d "6&/~/ + / _ / + Et" • The 
multijet signature has a complicated event topology. The final state contains two jets as- 
sociated with b quarks and 8 jets arising from hadronization of light quarks. In order to 
reconstruct the hh parentage of this signature, one would need excellent dijet mass resolu- 
tion to overcome combinatoric challenges. The fully-leptonic signature suffers from small 
branching ratios; in addition, the presence of multiple uncorrelated neutrinos make it cum- 
bersome to reconstruct the parentage of the final state leptons. Other signatures arising 
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from the ti W + W~ mode suffer from a combination of the challenges outlined in the full-jets 
and the fully-leptonic signatures. 

In addition to the charged-current decay modes of the hh, we do not consider signatures 
arising from the 66H°H° mode. Though the 6 b quark signature arising from the decay of 
each Higgs boson to b quarks is interesting, the 66H°Z° mode has a larger cross section and 
it leads to cleaner signatures. 

At the Tevatron, the most promising channels for the discovery of a down-type isosinglet 
quark are the bb Z°Z° and the bb H°Z° channels. These decay channels arise from the decay 
of each of the h quarks via a tree-level flavor- changing neutral interaction (mediated by H° 
or Z°) to a b quark. Jets associated with hadronized b quarks in the final state are powerful 
objects for analysis thanks to the 6-jet identification capabilities at both detector facilities 
(CDF and D0 ). 

To obtain a more realistic description of an hh event at the Tevatron, we decay the gauge 
bosons in the bb Z°Z° and 66H°Z° channels. We generate unweighted events for each of these 
primary decay modes, and we decay the gauge bosons to "final-state" SM particles (leptons, 
quarks, and gluons). When a Z° or a H° boson is decayed, we introduce a Gaussian smear 
on the reconstructed mass of the decay pair to approximate detector resolution effects. We 
take the dijet mass resolution, a(Mjj)/Mjj, of a jet pair to be 10% HQ- 

The energy 

resolution and transverse momentum resolution from Run I of the CDF detector are used to 
define the dilepton mass resolution from electrons and muons 39|. Using the signal events, 
we determine the fraction of events that pass a set of detection cuts. From this fraction and 
the bbZ°Z° and 66H°Z° cross sections, the cross section of hh events decaying to a particular 
signature is determined. 

In Table lTHl we present the angular and transverse momentum cuts imposed on the events. 
These cuts are applied at the parton level; the quarks and gluons have not been allowed to 
hadronize. In our notation, "/" and "j" refer to a light charged lepton (/ = e or fi) and a 
jet (j = u, c, d, s, g), respectively. 

In addition to the cuts delineated in Table EH dijet and dilepton mass cuts are applied. 
In signatures where a Z° boson decays to either a charged-lepton pair or a jet pair, the 
reconstructed mass of that pair should be close to the mass of the Z° boson. Therefore, we 
require the mass of a dijet /dilepton pair be within ±(0.1 x Mz). Complications may arise 
when more than two charged leptons or two jets are in the final state. In such instances, 
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TABLE III: Cuts applied to "final state" partons. I refers to either of the "light" charged leptons, 
e or /x. j refers to the light quarks and gluon - all of which would hadronize to form a jet. Using 
the "1" and "2" subscripts, we distinguish between b quarks arising from the primary decay of the 
h quark (b\) and from the subsequent decay of massive gauge bosons (62). 

Parameter Minimum Value Max Value 

-3.0 3.0 
25.0 
10.0 
0.4 



we define "Z°-like" lepton or jet pairs as those with mass closest to the Z° mass. In analogy 
to the Z° case, we impose a dijet reconstruction cut on the b jet decay products of the H° 
boson. Also, if a Higgs boson decays to a bb pair, there will be at least 4 b jets in the final 
state - two b quarks from the decay of the h quarks and two b quarks from the Higgs decay. 
Rather than requiring each b jet to have a transverse momentum greater than 25 GeV/c, 
we require that at least two non "H°-like" b jets have a transverse momentum greater than 
25 GeV/c. The remaining b jets must have a transverse momentum greater than 10 GeV/c. 

We also require additional cuts on the reconstructed h quark mass. For both the bb Z°Z° 
and the bbH°Z° modes, the h quark mass can be reconstructed from the appropriate b jet 
and gauge boson combinations. We constrain the invariant mass of a b quark and one of 
the Z°-like or H°-like final-state particle pairs be equal (within resolution) to the invariant 
mass of the other b quark and gauge boson decay products. In addition, each of these u h 
quark legs" must equal (within resolution) the desired h quark mass (e.g. 200, 250, or 300 
GeV/c 2 ). If one of the gauge bosons cannot be reconstructed (e.g. Z° decays to neutrinos), 
then the invariant mass of one of the b quarks and the other reconstructible Z°-like decay 
products must be equal (within resolution) to the h quark mass. 

With a set of formal cuts in place, we investigate specific final-state signatures arising 
from the 66Z°Z° and 66H°Z° channels. In Table ITVl we list five signatures for the bbZ°Z° 
mode and three signatures for the 66H°Z° mode. We consider signatures arising from Z° 
decays to a jet pair (jj), a 6-jet pair (bb), a neutrino pair (missing transverse energy, $ T ), 
and a pair of "light" charged leptons (I = e, //). Signatures resulting from the decay of the 
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TABLE IV: The signal cross section (in fb) for hh pair production and subsequent decay into 7 
final-state signatures. We use the '(-)z' notation to indicate that the quantities enclosed in the 
parentheses should have an invariant mass equal to the Z° boson mass. The cross sections are 
presented for three choices of the h quark mass (m h = 200, 250, and 300 GeV/c 2 ) and for two 
choices of the Higgs mass (Mh = 115 and 150 GeV/c 2 ). These cross sections assume perfect 6-jet 
tagging efficiencies, e& = 100%. 



Signature 




a (fb) 






a (fb) 




m h (GeV/c 2 ) 


200 


250 


300 


200 


250 


300 


M H (GeV/c 2 ) 


115 


115 


115 


150 


150 


150 


bb{jj)z(jj)z 


50 


13 


1.9 


79 


18 


0.90 


bb(jjMl+n z 


19 


4.9 


0.72 


28 


6.4 


0.30 


bb(jj) z $T 


90 


23 


3.2 


140 


32 


1.5 




15 


4.0 


0.55 


26 


5.6 


0.29 


bb(bb) z $T 


29 


7.3 


1.0 


44 


9.4 


0.45 


bb{bb)n(jj)z 


46 


15 


2.5 


4.9 


3.4 


0.43 


bb(bb)n(l + l-)z 


7.9 


2.9 


0.45 


0.74 


0.59 


0.080 


bb(bb) H tf T 


37 


12 


1.9 


4.0 


2.3 


0.36 



Higgs boson to a 6-jet pair are also considered. The cross sections contained in Table ITVl do 
not account for 6-tagging efficiencies [e& = 100%]. In Section ITV D 21 we loosen this constraint 
to determine estimates of the h quark mass reach at the Tevatron. 

As seen in Table IIV[ the most promising decay signatures in the bb Z°Z° channel are 
bb(jj) z (jj)z, bb(jj)z0fa)z, and bb(jj) z (l + l~)z- These decay signatures benefit from at 
least one of the Z° bosons decaying hadronically. The mode in which both Z° bosons decay 
to light quarks (full-jets mode) appears to provide the best reach in h quark mass. The 
bb(jj)z0fir)z an d bb(jj)z(l + l~)z signatures are relatively clean with slightly diminished 
signal. If the h quark mass is 250 GeV/c 2 and the mass of the Higgs boson is 150 GeV/c 2 , 
one expects to produce (18, 180) bb(jj)z{jj)z events, (6.4, 64) bb(jj)z(l + l~)z events, and 
(32, 320) bb(jj)z$T events in (1, 10) fb" 1 of data at the Tevatron. Many of the 66Z°Z° 
signatures are interesting and are similar to the signatures of ti production at the Tevatron. 
In Section lTV D 2l we find that some of these signatures are closely related to the ti production 
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signal, resulting in large backgrounds. 

Next, we consider final state signatures arising from the 66H°Z° channel. In this mode the 
Higgs boson is produced via the tree-level FCNC decay of the h quark. Since the strength of 
the coupling to the h quark is proportional to the product of the flavor- changing interaction 
and the mass of the h quark [see Fig. the Higgs coupling to the h quark may be sizable. 
This leads to an intriguing scenario in which the discovery of an isosinglet quark may lead to 
the discovery of the Higgs boson. We consider signatures arising from the decay of the Higgs 
boson to b quarks. Decays of the Higgs boson to W ± pairs lead to complicated signatures 
that are not ideal for an h search at the Tevatron. 

In Table |WJ we consider three signatures arising from the bbH°Z° channel. The Z° boson 
decays to either a quark/ ant i- quark pair, a pair of charged leptons, or a pair of neutrinos. 
As discussed above, jets in the final state result from the hadronization of the light quarks, 
and missing transverse energy (Et) results from the decay of a Z° boson to neutrinos. 

As expected, the branching ratio of the Higgs boson to a b quark pair is significantly 
reduced as one increases the Higgs boson mass from 115 GeV/c 2 to 150 GeV/c 2 . Signatures 
arising from the 66H°Z° channel have the most reach when there is a light Higgs boson. In 
addition, the bb(bb)u_(jj)z and the &&(5&) h Et signatures have much larger cross sections than 
the bb(bb)ti(l + l~)z signature. If the h quark mass is 250 GeV/c 2 and the Higgs mass is 115 
GeV/c 2 , one expects to produce (15, 150) bb(bb)u(jj)z events and (12, 120) &&(&5) h Et in (1, 
10) fb _1 of data at the Fermilab Tevatron. 

2. Backgrounds 

In the previous section we presented a number of hh signatures and their expected tree- 
level cross sections at the Tevatron. To ascertain the h quark mass reach at the Tevatron, 
we need to understand the backgrounds associated with these signatures [see Table HVl . 

n 

We use the software package MadEvent 40] to study the SM backgrounds for these 
signatures. MadEvent generates and calculates the tree-level contributions to a given process 
(parton-level calculation) using helicity amplitude methods [41]. We do not consider one- 
loop contributions to the background. Though FCNCs occur at one-loop in the SM, we 
expect their contribution to the background to be small. If one includes one-loop processes 
in the SM background calculation, the one-loop contribution to the ISVLQ model should be 
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included in the signal calculation. 

At the parton level, each of the hh signatures contain six final state particles (quarks and 
leptons). Though MadEvent can generate the diagrams and associated matrix element for 
a signature containing six "final-state" particles, the evaluation of some matrix elements is 
computationally intractable. In particular, the quantum chromodynamic backgrounds to the 
bb(jj)z(jj)z and the bb(bb)u(jj)z signatures consist of an enormous set of tree-level diagrams 
that overwhelm most computing clusters. Therefore, when computationally feasible, we use 
MadEvent to calculate the SM background to the hh signatures. The order of the background 
calculation will accompany all estimates of the background cross sections. For signatures 
like the bb(jj)z{jj)z, in which the matrix element for the backgrounds are not calculable 
using current technology, one would need to use approximate methods or to measure the 
background from the Tevatron data itself. 

For each signature, we require background processes to pass the cuts outlined in Table ITTT1 
and the invariant mass cuts discussed in Section IIVD II As with the signal events, we 
assume that at the Tevatron the dijet mass resolution can be approximated by a Gaussian 
distribution with a resolution of 10% of the invariant mass. The dijet and dilepton cuts and 
the cuts in Table IIHI are designed to reduce the size of the background. For example, the 
hard cut on the transverse momentum of b quarks from the primary h quark decay (p b T > 25 
GeV/c 2 ) in conjunction with the jet separation cut (AR > 0.4) helps reduce backgrounds 
from gluon splitting to bb. The dijet mass cuts help reduce QCD backgrounds that duplicate 
our hh production signatures. 

In Table we present the results of our SM background calculation for each of the sig- 
natures listed in Table IIV1 For the SM background calculations we assume a 6-jet tagging 
efficiency of 50%; if a b jet is not tagged, then it is treated as a jet (j). For signatures con- 
taining 2 or 4 b jets, we require at least 1 or 3 6-jet tags, respectively. As discussed above, we 
are unable to calculate the matrix element of the backgrounds for some signatures because of 
computational limitations. In columns four and five of Table W\ we indicate the electroweak 
order of the background for which we were able and unable to calculate. In these columns, 
a number indicates the number of electroweak vertices in the background calculation, the 
"FT" indicates the calculation is a full tree-level calculation, and the "*" indicates that 
the background is forced to produce massive gauge bosons before decaying to the indicated 
signature. For example, we calculated the bb(jj)z{l + l~)z background originating from di- 
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TABLE V: Standard Model background to the hh pair production signatures outlined in Sec- 
tion IIV D II The SM background cross sections are presented for a Higgs mass of 115 and of 150 
GeV/c 2 . A 6-jet tagging efficiency of 50% is used and at least 1 or 3 b tags are required for sig- 
natures containing 2 or 4 6 jets, respectively. In the "EW Order" columns we indicate the order 
of each of the electroweak calculations. A number in either of these columns indicates the number 
of electroweak vertices in the background calculation, the "FT" indicates the calculation is a full 
tree-level calculation, and the "*" indicates the background is forced to produce massive gauge 
bosons before decaying to the indicated signature. 



Signature 




Cfefcgrnd(fb) 






&bkgrnd{Vo) 




EW Order 


M h (GeV/c 2 ) 


200 


250 


300 


200 


250 


300 


(calc) 


(uncalc) 


M H (GeV/c 2 ) 


115 


115 


115 


150 


150 


150 






bb(jj)z(jj)z 


110 


56 


16 


110 


57 


15 


2,4* 


0,> 4 


bb(jj)z(l+r)z 


0.023 


0.0098 


0.0044 


0.024 


0.010 


0.0043 


2,4* 


> 4 


bb(jj) z $T 


1.8 


0.57 


0.30 


1.3 


0.57 


0.53 


2,4* 


> 4 


w>(j+r)z#r 


8.4 


6.1 


2.6 


8.4 


6.1 


2.6 


FT 




bb(bb) z $T 


0.023 


0.014 


0.0081 


0.022 


0.014 


0.0079 


2,4* 


> 4 


bb{bb)u(jj)z 


0.059 


0.0023 


0.0010 


0.0025 


0.0019 


0.00091 


2,4* 


0,> 4 


bb{bb)n{l + l-)z 


0.0035 


0.0013 


0.00047 


0.0026 


0.0015 


0.00047 


2,4* 


> 4 


&&(W>)h#t 


0.021 


0.014 


0.0087 


0.013 


0.013 


0.0090 


2,4* 


> 4 



agrams with two electroweak vertices and with four electroweak vertices. The background 
with two electroweak vertices includes diagrams where the jets do not come from Z° decay. 
The background with four electroweak vertices includes all bb(jj)z(l + l~)z signatures that 
arise from bbXX , where X is a massive gauge boson, X = Z°, W ± , or H°. 

For most of the signatures discussed in Section IIV D 11 the SM background appears to 
be manageable when compared to the signal cross sections [see Table ITVj . The notable 
exceptions are the bb(jj)z(jj)z and the bb(l + l~)z^T signatures. The large backgrounds for 
these signatures (relative to the other signatures) can be traced to top quark pair produc- 
tion. At the Tevatron, tt production, like the hh production, proceeds via quark/anti-quark 
annihilation and gluon-gluon fusion. Once the tt are produced, each top quark decays to a 
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b and a W + . Subsequent decay of the W 1 * 1 bosons results in the following three signatures: 
u bb{jj)w{jj)w" (full-jets), u bb{jj) w Pfa" (semi-leptonic), and u bbl + l~^ T " (fully leptonic) . 
We use the '(-)w' notation to indicate that the quantities enclosed in the parentheses have 
an invariant mass equal to the W ± boson mass. 

If the dijet mass resolution at the Tevatron were perfect, then a jet pair from a decay 
and a jet pair from a Z° decay would always be distinguishable. However, the dijet mass 
resolution at the Tevatron is not perfect; therefore, a fraction of the ti events decaying to the 
full-jets mode will mimic the bb(jj)z{jj)z signature. We find that for a Higgs mass of 150 
GeV/c 2 and an h quark mass of 250 GeV/c 2 , the background to the signature bb(jj)z(jj)z 
is 57 fb. The purely QCD component of this background (denoted "0" in Table 0) was 
not calculated. Though one expects the invariant mass cuts to substantially reduce this 
background, it is unlikely that it will be negligible. Thus we conclude that the bb(jj)z(jj)z 
signature is background limited and that other hh signatures will provide a better h quark 
mass reach. However, if nature contains an h quark with a mass less than ~ 250 GeV/c 2 , 
this signature will provide a channel to measure the mass of the h quark (h quark mass 
peak). 

The bb(l + l~)ztf>T signature also has a sizable ti background component. When a ti event 
decays to the fully leptonic mode, the invariant mass of the two charged leptons may be 
close to the mass of the Z° boson. One concludes that, like the bb(jj)z(jj)z signature, the 
h quark mass reach of the bb(l + l~)ztf> r r signature is diminished because of the background. 

The bb(H)z{l + l~)z and bb(jj)ztf>T signatures provide clean alternatives to the full jets 
signature. Unlike the bb(jj)z(jj)z signature, the bb(jj)ztf>T signature is not afflicted by a 
large ti background. The semi-leptonic decay of a ti event can duplicate the bb(jj)ztf>T 
signature if the dijet mass is close to the Z° mass and the charged lepton is undetected. 
In Table we include this and other backgrounds to the bb(jj)z^T signature in which a 
charged lepton from the decay of a W 1 * 1 is undetected. In order to determine the "unde- 
tected lepton" background, we assume that this background originates from events in which 
the charged lepton travels through an uninstrumented region of the detector J2|. The 
"lost-lepton" background is also included in the bb(l + l~)ztf>T, bb(bb)ztf>T, and bb(bb)H^T sig- 
natures. Though other experimental issues, like jet energy mismeasurement of QCD jets, 
are likely to increase the background, we do not include these in our calculation. 

The bb(jj)z(l + l~)z signature also avoids large ti backgrounds. The semi-leptonic decay 
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of a tt event can mimic this signature if the dijet mass is Z°-like and the detector spuriously 
identifies an additional lepton. The dilepton mass of the real and spurious charged leptons 
must be Z°-like. This component of the background is expected to be small; therefore, we 
do not include it in the background estimate. 

Since the bb(jj)z(l + l~)z and the bb(jj)ztf>T are clean signatures with small backgrounds, 
we expect these modes to provide the greatest reach at the Tevatron. In (1, 10) fb _1 of data, 
the highest h quark mass accessible by the bb(jj)z(l + l~)z and the bb(jj)ztf>T signatures are 
(230, 290) GeV/c 2 and (270, 320) GeV/c 2 , respectively 

While the 66Z°Z° mode is likely to provide the best reach for an h quark search, the 
bbH°Z° mode provides an opportunity to discover the Higgs boson in conjunction with the 
h quark. If nature favors a light Higgs boson (e.g. Mh = 115 GeV/c 2 ), the dominant 
branching ratio of the Higgs boson is the bb mode, BR(H° — > bb) = 73.2%. However, if the 
mass of the Higgs boson is larger than current electroweak best fits (43J, the 66H°Z° mode 
is less powerful. For example, at a Higgs mass of 150 GeV/c 2 , the Higgs branching ratio to 
bb is 17.6%. 

Unlike many of the bb Z°Z° signatures, the bb H°Z° signatures do not suffer from the large 
ti background. The reduction in the tt background is primarily because of the bb signature 
from the Higgs decay (W ± can not decay to a bb pair). Though these signatures have low 
SM backgrounds, 6-tagging efficiencies will reduce the expected signal cross section. 

Standard Model background to each of the three bbH°Z° signatures is small [see Ta- 
bleEl- Because of computational limitations, the pure QCD component of the bb(bb)H(jj)z 
background (denoted "0" in Table IVj) was not calculated. As with the background for the 
bb(jj)z(jj)z signature, the QCD component of the bb(bb)H(jj)z signature will need to be 
measured from the data. It is unlikely that the background from these processes will be 
large. Assuming a 6-tagging efficiency of 50% and requiring that three of the four 6-jets 
is tagged, for a Higgs mass of 115 GeV/c 2 one expects the mass reach in the bb(bb)H^T 
to be (230, 290) GeV/c 2 in (1, 10) fb _1 of data. Moreover, because the background to the 
bb(j j) h(j j) z signature is not expected to be large, we expect for a Higgs mass of 115 GeV/c 2 
the h mass reach of this mode to be ~ (220, 270) GeV/c 2 . 
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V. hh PRODUCTION AND DECAY AT THE LHC 



At the CM energy of the CERN LHC (-^/s = 14 TeV), hh pair production is dominated 
by the gluon-gluon fusion subprocess. Contributions from subprocesses in which a valence 
quark from one proton and its anti-particle from the other proton (sea quark) annihilate via 
the strong interactions are important but sub-dominate to the gluon-gluon fusion. 

For the LHC we consider the same primary decay channels of the hh pair as at the 
Tevatron: tfW+W", 66Z°Z°, 66H°H°, tbW~Z°, 66H°Z°, and tlW-R . We also impose the 
same set of loose cuts on the primary decay products of the h quark [see Table ITT] . This 
ensures that b quarks produced from the primary decay of the hh pair conform to basic 
geometry and event selection requirements of the detectors. 

The cross sections for hh production and (primary h quark) decay at the LHC are shown 
in Fig. In this figure, the new mixing parameter, £, is set to 1, the Higgs mass, Mff, is set 
to 150 GeV/c 2 , and the h quark mass, m^, runs from 100 - 3000 GeV/c 2 . The cross sections 
for the hh primary decay modes at the LHC are roughly two orders of magnitude larger 
than the corresponding cross sections at the Tevatron. Below 300 GeV/c 2 , the bbZ°Z° mode 
is once again the dominant primary decay mode of hh pair production. Above an h quark 
mass of 300 GeV/c 2 , each of the primary decay cross sections are comparable; however, the 
decay modes with at least one charged-current decay of the h quark tend to be larger. 

At the LHC the cross sections for each of the three charged-current primary decay modes 
are significant, unlike those at the Tevatron. For an h quark mass of 500 GeV/c 2 and a 
mixing parameter £ = 1, the LHC should produce ~ 20 000 hh events decaying to the 
tt W + W~ mode in 100 fb _1 of data. The charged-current mode of hh decay is important 
because our parametrization of the L d (CKM) matrix in the ISVLQ model predicts a rela- 
tionship between the size of charged-current and neutral-current interactions. As discussed 
in Section IIV D 21 the charged-current modes have final states that are more complicated 
than those encountered in FCNC decay modes. Additional complications are attributed to 
the decay of the top quark resulting in an additional W ± boson. With additional particles 
in the final state, dijet mass resolution and the calculation of signature backgrounds are 
extremely important components for an analysis. Rather than address the charged-current 
modes in this paper, we defer this analysis to future research when the difficulties of the 
background calculation and details about dijet mass resolution and 6-tagging can be better 
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resolved. 

In our analysis of hh production at the Tevatron [see Section TlV Dj . we found that signa- 
tures of the bb Z°Z° mode provide the highest h quark mass reach. We conclude that in 10 
fb -1 of data, hh pair production can be probed up to an h quark mass of 320 GeV/c 2 . At 
this h quark mass and for a Higgs mass of 150 GeV/ c 2 , the cross section to the bb Z°Z° mode 
{o'bbzz = 20 fb) and the branching ratio to the "optimal" signatures combine to produce 
a handful of signal events with negligible background. Based on our study of the bbZ°Z° 
mode at the Tevatron, we infer a mass reach for this decay mode at the LHC We assume 
that the branching ratios of the bb Z°Z° mode to final-state signatures and the detection of 
these signatures at the LHC are similar to the Tevatron. Therefore, at the LHC, the upper 
limit on the h quark mass is encountered when approximately 200 events are produced in 
the 66Z°Z° mode. The expected integrated luminosity at the LHC is 100 fb -1 ; thus one can 
probe the bb Z°Z° cross section down to 2 fb. This corresponds to an h quark mass reach of 
1100 GeV/c 2 [see Fig. Eg. 
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VI. CONCLUSION 



We have investigated an E 6 -inspired extension of the Standard Model in which an exotic 
charge —1/3 isosinglet vector-like quark (denoted h) interacts predominantly with the third 
generation of quarks. In this model, the CKM matrix is no longer unitary and it is replaced 
by a 3 x 4 matrix containing new angles and phases. The loss of CKM unitarity is accom- 
panied by the emergence of tree-level flavor- changing neutral currents mediated by both Z° 
and H° bosons. Flavor- changing neutral-current interactions between the h quark and the 
b quark produce signatures of hh production accessible for detection at hadron colliders. 

At the Fermilab Tevatron, we find that h quark discovery through pair production is 
accessible up to an h quark mass of (270, 320) GeV/c 2 in (1, 10) fb _1 of data. Previous b' 
analyses from Run I of the Tevatron were used to infer that an h quark is currently excluded 
up to 200 GeV/c 2 . The (270, 320) GeV/c 2 mass reach is attainable through the decay of 
the 66Z°Z° mode to the bb(jj)z(l + l~)z or the bb(jj)z^T signatures. Furthermore, primary 
decay of an hh pair to the 66H°Z° mode provides the opportunity for the discovery of the 
Higgs boson in conjunction with h quark discovery. The viability of the 66H°Z° mode at 
the Tevatron hinges on the branching ratio of the Higgs boson to a b quark pair. 

At the CERN LHC, hh pair production is accessible through both charged-current and 
neutral-current decays of the h quark. In 100 fb _1 of data, we find that the h quark mass 
reach through the bbZ°Z° primary decay mode is 1100 GeV/c 2 . To understand the reach 
of the charged-current primary decay modes, an analysis of potential signatures, the effect 
of detector limitations, and signature backgrounds is necessary. A thorough analysis of hh 
charged-current decay modes (t6W~Z°, £6W~H°, and ttW + W~) at the LHC is deferred to 
future work. 
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